DOCUMENT RESUME 



ED 467 188 



PS 030 575 



AUTHOR 

TITLE 

INSTITUTION 

SPONS AGENCY 
PUB DATE 
NOTE 

AVAILABLE FROM 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Sutherland, Carol; McCroskey, Jacquelyn; Halfon, Neal 
The Challenges of Measuring the Impact of Proposition 10. 
Building Community Systems for Young Children. 

California Univ., Los Angeles. Center for Healthier Children, 
Families and Communities. 

David and Lucile Packard Foundation, Los Altos, CA. 

2001-08-00 
57p . 

UCLA Center for Healthier Children, Families and Communities, 
Box 951772, Los Angeles, CA 90095-1772 ($5). Tel: 310-206- 
1898; Fax: 310-206-3180. For full text: 
http : / /healthychild . ucla . edu . 

Guides - Non-Classroom (055) — Reports - Descriptive (141) 
EDRS Price MF01/PC03 Plus Postage. 

Counties; ^Evaluation Criteria; Evaluation Methods; 
^Evaluation Problems; ^Family Programs; Program Evaluation; 
Well Being 

California; Family Support; ^Indicators 



ABSTRACT 

In compliance with California’s Proposition 10 (Children and 
Families First Act) , county commissions are being asked to monitor what 
differences the family support programs they are funding are making in the 
lives of the children and families they serve. Each commission has the 
responsibility of selecting a set of indicators to measure the impact of 
Proposition 10 on the lives of all children from birth to age 5 and their 
families, as well as performance measures to assess the changes in the client 
population targeted. This report provides a historical overview of how 
indicators have been used to measure child and family well-being and recent 
innovations in monitoring strategies necessitated by results-based decision 
making. The report also discusses approaches that can be used to facilitate 
the indicator selection process, and offers recommendations for funding and 
using results-based decision making to build effective monitoring efforts. 
Following an introduction, the report reviews research to date on indicators 
and results-based decision making. The next section of the report explores 
use of results, indicators, and performance measures, including language 
issues, use of a conceptual framework, and criteria for indicator selection. 
Issues of integrating and coordinating data collection are then explored. The 
report concludes with recommendations on funding and using results-based 
decision making, including investing in a county-wide and state-wide data 
infrastructure, and developing a research agenda. The report’s nine 
appendices include a language translation key for Proposition 10 guidelines 
terminology; examples of indicators of short-term results; a matrix of 
existing child well-being indicators; and examples of indicators related to 
service integration, accessibility, and cultural appropriateness. (Contains 
41 references.) (HTH) 
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The Challenges of Measuring the Impact of Proposition 10 



I. Introduction 



The California Children and Families Act (Proposition 10) provides a historic opportunity to 
positively affect California’s youngest children and their families. The funds available to counties 
from tobacco tax revenue are earmarked for programs and services for children from the prenatal 
period to age 5, a time of critical importance for their successful physical, emotional and social 
development. The goal of the Children and Families Act is to provide the resources and support 
families need so that their children are healthy and ready to succeed by the time they enter 
kindergarten. The current array of programs and services for children and families is often 
fragmented, difficult to access or non-existent for the youngest ages, and tends to focus on providing 
services, but not necessarily on tracking results. Proposition 10 offers planners, policymakers, 
service providers and communities an opportunity to go beyond “business as usual”) to create a 
comprehensive and integrated system of support and care for families with young children which is 
family-friendly, culturally sensitive and accountable. 

To receive their allotment of funds from the California Children and Families State Commission, 
count Proposition 10 commissions had to submit an initial strategic plan and each year must review 
and update this plan. As part of their planning process, county commissions are required by the act 
to identify not only the programs, services and projects they will fund to accomplish their goals, but 
also how they will determine the success of these programs, services and projects using appropriate, 
reliable measures. 1 

Most service providers for children and families are accustomed to using data after programs have 
been implemented to evaluate their success, and often success is measured in terms of the quantity 
of services delivered. However, Proposition 10 commissions are being asked to do something 
different — to monitor what difference the programs they are funding are making in the lives of the 
children and families they serve. The Children and Families First Act requires each commission to 
identify the results they hope to achieve first and to develop a “results accountability framework” 2 
to show how they will monitor the impact of their funded strategies on these desired results. This 
approach requires that Proposition 10 commissioners and planners begin the planning process not 
by deciding what programs they will fund, but by defining desired results (e.g., all children enter 
school healthy and ready to learn). They will then select indicators to measure progress toward the 
desired results (e.g., the percentage of children fully immunized by age 2, the percentage of children 
in quality day care), and then shape programmatic strategies based on “what works” to achieve those 
desired results (e.g., home visitation for new mothers, training of child care staff, creation of family 
resource centers, ensuring access to health care for all children, etc.). Thus, the focus shifts from an 

1 California Children and Families Commission, Guidelines: A Resource for Developing Proposition 1 0 Strategic Plans, 
1999. 

2 For a full description of the results accountability framework, see a companion brief in this series: Friedman, M. Results 
Accountability for Proposition 10 Commissions: A Planning Guide for Improving the Well-Being of Young Children 
and Their Families, in N Halfon, E Shulman, M Hochstein and M Shannon, eds., Building Community Systems for 
Young Children, UCLA Center for Healthier Children, Families and Communities, 2000. 
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emphasis on the services provided — “What activities should be carried out? What should be 
done?” — to a different consideration: “Ultimately, what results, what impact, what changes in the 
conditions of children and families do we expect to see from the activities performed?” 

In order to monitor progress towards the desired results, each county commission has the 
responsibility of selecting a set of indicators to measure the impact of Proposition 10 on the lives of 
all children 0-5 and their families, as well as performance measures to assess the changes in the 
client populations targeted by specific programs, services and projects. Results from a survey 3 of 54 
of the 58 counties in California indicate that the vast majority of counties are focusing on the four 
results areas identified by the State Commission Guidelines : Improved Family Functioning (98%), 
Improved Child Development (98%), Improved Child Health (98%), and Service Integration (85%). 
However, most counties (59%) have not identified a set of indicators to track progress towards those 
results. Many reported indicator selection, standardization and collection as a major challenge, and 
many are waiting for the state reporting requirements to be developed before fully developing their 
own evaluation systems. A large number of counties (44%) do not yet have an internal or external 
evaluator. 

The California Children and Families Commission (CCFC) is in the process of developing two tools 
to assist the counties in their monitoring and evaluation efforts. The first, the County Results and 
Indicators Data Collection Tool, is still being finalized. This tool draws heavily from the State 
Commission’s Results Document and site visits conducted by CCFC and SRI International to six 
counties beginning their data collection efforts. It is designed as a first step toward establishing a 
common set of indicators that could be used to measure the results across counties. It is designed 
to be used with grantees to collect basic information on short-and long-term results, strategies being 
implemented, and populations being served. 

The second, the Annual Report, was developed collaboratively by CCFC and the California Children 
and Families Association (CCAFA) to assist the counties in reporting their progress to CCFC. The 
first year’s report (2001) is designed to standardize the format in which information is submitted to 
CCFC. County commissions have an opportunity to describe their major accomplishments and the 
current status of their reporting and data collection efforts for strategies, results and indicators. The 
second year report (2002) will incorporate more detailed information on the population being served 
(i.e., ethnicity, special needs, teen mothers, etc.) as well as the specific indicators of progress towards 
those results. 

In addition to tracking the success of their funded programs, county commissions must also 
determine how programs, services and projects relating to early childhood development will be 
integrated into a consumer-oriented and easily accessible system. Data development, collection and 
analysis is exceedingly important for monitoring and demonstrating results, and is a key strategy for 
providing the information base that is essential for building integrated service systems for children 
and families. 



3 California Children and Families Commission Research and Evaluation Unit, “An Overview of Three Major Evaluation 
Initiatives.” Presentation, May 2, 2001 . 
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In summary, the county commission’s charge related to monitoring includes: 

> tracking outcomes for the whole population of families with young children to monitor 
progress toward desired results and to inform the strategic planning process; 

> monitoring the effectiveness of specific programs and using that information for continuous 
quality improvement of the programs and to inform the commission’s funding decisions; and 

> measuring overall progress towards integration of services and systems. 

This paper will begin with a historical overview of how indicators have been used to measure child 
and family well-being and recent innovations in monitoring strategies necessitated by results-based 
decision making. We will then discuss the common sources of language confusion in the use of 
indicators and performance measures; approaches that can be used to facilitate the indicator selection 
process; and the range of traditional and new measures of child health and development that might 
be considered useful in monitoring child and family well-being. We will then know how outcomes 
and indicators can be used for planning and monitoring efforts; the challenges inherent in using 
existing information systems, and the opportunities for developing new data sources and creating 
a data infrastructure. Last, we will turn to the importance of communicating the information 
contained in the data in a way that “tells the story” of Proposition 10 to many different constituencies 
with different perspectives, values, assumptions and attitudes, and identify tools that can be used to 
facilitate this process. We will conclude with recommendations for how to fund and use results based 
decision-making to build effective monitoring efforts. 



II. Research to Date on Indicators and Results-Based Decision Making 

Indicators of child well-being 

In The Measure of Reality, Alfred W. Crosby asserts that the “distinctive intellectual 
accomplishment” of Western Europe during the Middle Ages was to “bring mathematics and 
measurement together and to hold them to the task of making sense of a sensorially perceivable 
reality” (Crosby 1997: 17). It is certainly true that we in the West have great faith in quantification, 
so much so that we tend to doubt the reality of that which cannot be quantified. The quantification 
of social conditions poses so many challenges, however, that no fully satisfactory and universally 
accepted method has emerged. 

In America, the New Frontier and Great Society programs of the 1 960s and 1 970s raised the question 
of how to quantify social well-being in relation to “social indicators” that could track the impact of 
technological progress on social conditions (Shonfield & Shaw 1972). Government agencies and 
public-interest groups have published reports analyzing trends in indicators over time (Andrews 
1996; Dear 1996; U. S. Department of Health and Human Services 1998). Advocates for children 
and families readily adopted this notion, using such indicators to measure changes in the quality of 
life for children, youth and families (Lasch & Siegel 1976; Wald & Kirst 1989). 
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Initially, indicators about children’s lives were largely based on statistics about mortality and 
survival. Over time, measures of disease, disability and dysfunction have been added. More 
recently, with widespread adoption of such concepts as “resiliency” and “protective factors,” there 
has been increasing interest in developing new indicators to measure the assets and capacities of 
children, families and communities, in addition to needs, to present a more balanced picture of 
children’s well-being (Ben-Arieh 2000; Search Institute). 

One of the defining influences on children’s policy development in the 1970s and early 1980s was 
the emergence of the Children’s Defense Fund as a prominent voice for children and CDF’s use of 
national and state data to profile the needs of America’s children (Halfon et al. 1998). In 1984, CDF 
began issuing a yearly special report entitled The Health of America 's Children: The Maternal and 
Child Health Data Book, along with other reports that described a wide range of health and social 
indicators for the country as a whole, as well as for individual states. 

During the early 1980s, children’s councils, advisory boards, and planning agencies were established 
in many states and communities to monitor and publicize the conditions of children’s lives (Smecker 
1989). Prominent examples of these were found in Ohio and Illinois, and in cities such as New York, 
Los Angeles, Chicago and San Francisco (Halfon et al. 1998). The State of the Child Report 
(Illinois) was a three-volume report issued over several years that set the stage for others that would 
follow, including the Conditions of Children report in California (Wald & Kirst 1989). 

In the early 1990s, Children Now, a California children’s advocacy organization, produced a 
statewide children’s report in an entirely new format, the “report card,” using letter grades. More 
recently, a number of children’s “scorecards” have emerged that use indicators to track and compare 
community conditions for children and families (Gardner 1994; Santa Cruz County Council for At- 
Risk Youth 1990; Orange County Children’s Services Coordination Committee 1999; Kids 
Network/UCSB Department of Education 1998; United Way of Greater Los Angeles and Los 
Angeles County Children’s Planning Council 1999; County of San Diego Health and Human 
Services Agency 2000.) A resurgence of interest in using such indicators was marked by a 1994 
national conference on the intellectual and practical challenges of using indicators of children’s well- 
being (Hauser, Brown & Prosser 1 997). A recent inventory of child, youth and family indicator 
projects found over 90 such efforts across the country (Child Trends 2000). 

KIDS COUNT, sponsored by the Annie E. Casey Foundation, is perhaps the best-known national 
report on child and family indicators. The foundation funds groups in all 50 states to annually 
produce comparable data on a core set of indicators. These statewide scorecards have helped to 
familiarize a large number of people with outcomes and indicators, sources of available data, some 
of the technical issues involved in trying to track change over time, and efforts to use indicator data 
in advocating to improve the conditions of children. 

Performance measures 

While indicator reports such as those described above present a profile of health and well-being for 
the whole population, performance monitoring uses measures to systematically assess the impact of 
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particular services on a particular target population. The use of performance measurement has grown 
considerably over the past three decades. Performance monitoring originated in the business world, 
has subsequently been adopted in health care and social services, and is now being more widely used 
for government programs (Institute of Medicine 2001). 

Indeed, interest in performance measurement has increased during the last decade at every level of 
government (Institute of Medicine 1 997). At the federal level, the Government Performance and 
Results Act of 1993 required regular performance reporting by all federal agencies. A recent study 
found that 47 of the 50 states use results-based budgeting techniques with associated performance 
measures (Melkers & Willoughby 1998). Another study found that about 40 percent of 
municipalities not only report that they use such measures, but the authors believe that they make 
“meaningful use of performance measures in their management and decision processes” (Poister & 
Streib 1999: 332). 

This widespread use of performance measurement has resulted in the development and adoption of 
a range of techniques for continuous quality improvement based on outcomes and results based 
accountability. These new measurement techniques has influenced how factory managers manage 
production, how hospital administrators evaluate the performance of their cardiac surgery team, and 
how policymakers examine the potential and results of different legislative actions. Some of these 
performance-monitoring activities have been formalized into systems of ongoing measurement and 
accountability. For example, in the health care arena, the National Committee for Quality Assurance 
(NCQA) has created a measurement set called the Health Employers Data Information System 
(HEDIS) that is part of a performance monitoring process widely used throughout the United States. 
In fact, all state Medicaid programs use the HEDIS monitoring system to evaluate the performance 
of their managed care contracts. For children, Medicaid HEDIS measures include the receipt of 
well-child care and immunizations. Recently, HEDIS has adopted a new survey to monitor parents’ 
experience and evaluation of health care for children with chronic and disabling medical conditions. 

Connecting these developments, many researchers and government agencies are now working to 
identify indicators that would not only measure the performance of social programs (their 
compliance with rules and regulations), but would also show how these programs help to improve 
the lives of families and children through results accountability. Proposition 10 requires a results 
accountability approach to guide commissions “through dynamic cycles of planning, allocation, 
implementation and evaluation” (California Children and Families Commission 1999). State and 
county commissions can build on the lessons learned from previous efforts - combining knowledge 
about how to identify meaningful indicators, share information with the public, track performance 
measures over time, and assess the impact of services — in order to most strategically improve 
conditions for young children and their families. 

While the results accountability model adopted by the commission is a promising approach, it is still 
in development. There are a number of examples from around the country cited as “turn the curve” 
success stories (where interventions have changed the direction of a trend, turning it toward desired 
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results). 4 However, not much evidence for the effectiveness of results accountability is available in 
published studies. One of the challenges of the results accountability approach is the difficulty of 
achieving accountability as one moves from individual programs to sets of programs or integrated 
systems, where determination of “causes” is much more complex. Another challenge is to shift the 
focus from process measures (e.g., services offered) to “true” outcome measures (e.g., the well-being 
of children and families). The experiences of Proposition 10 county commissions in implementing 
this approach to results accountability will provide a valuable testing ground and feedback on its 
usefulness from a broad range of contexts, including different populations and varying resources, 
in California’s 58 counties. 



III. Using Results. Indicators and Performance Measures 

Language issues 

A potential pitfall of using the results accountability framework is the confusion that can arise when 
different terminology is used by different people to refer to the same underlying concepts. Counties 
should use terms that make the most sense to them. However, it is important that, once those terms 
are agreed upon, they be used consistently. Appendix A provides a crosswalk between terms and 
definitions related to results accountability used in the State Commission Guidelines and some 
corresponding terms commonly used by others involved in results accountability and strategic 
planning. 5 

In this paper, we use the term indicator to refer to measure of the extent to which results (i.e., what 
the county as a whole is striving for) are being achieved with regard to the whole population. For 
the purposes of Proposition 10, each commission will select a relatively short set of indicators to 
reflect the health and well-being of all children and families in the county, with an emphasis on 
results that are being targeted by Proposition 10 initiatives. Performance measure refers to those 
measures used to determine whether programs, services or projects (. strategies ) are achieving their 
program results with the targeted population (what the funded community agencies will be held 
accountable for). Performance measures include both the quantity of services provided (e.g., number 
of women served by a residential treatment program for substance-abusing pregnant women) as well 
as the quality of those services (e.g., percent of babies bom drug-free among the women participating 
in the treatment program). 

The distinction between measures of the status of the whole population vs. measures of the 
performance of specific agencies/programs is an important one and, if not clearly articulated, tends 



4 For examples of Turn the Curve Success Stories, see The Results and Performance Accountability Implementation 
Guide at www.RAGuide.org. 

5 A related “crosswalk” with a somewhat different orientation can be found in Friedman, M. Results Accountability for 
Proposition JO Commissions: A Planning Guide for Improving the Well-Being of Young Children and Their Families , 
in N Halfon, E Shulman, M Hochstein and M Shannon, eds.. Building Community Systems for Young Children, UCLA 
Center for Healthier Children, Families and Communities, 2000. 



to create confusion for planners. A results accountability framework should include both types of 
measures. For some measures, there may be overlap in the indicators/performance measures selected 
to track both program and population results (e.g., low birthweight can be tracked for the population 
as a whole as well as for the recipients of specific prenatal services); however, for the population as 
a whole, the indicator will refer to results for the entire population, whereas for a specific program 
or service, the indicator will refer to a subset of the whole population — those members of the whole 
population who have participated in a particular service or program. 

In performance monitoring, it is important to remember that agencies are accountable for their own 
performance with the families they see, not their impact (or lack thereof) on the whole population. 
Even the most successful agency or program will not be able to improve conditions in the whole 
population by itself. For example, an agency may be successful in preventing crime among youth 
in their target population, but the overall crime rates in the community served or in the county may 
not improve or may even worsen. Possible explanations for the lack of an impact on the whole 
population are: not enough programs are covering enough of the population to work on the issue 
effectively; more time is needed for improvement to show in the statistics; the measures used to 
discern changes are not sensitive or appropriate; or the intervention was not focused enough on the 
key causes of the problem. On the other hand, if an improvement is noticed in the whole population, 
it will often be difficult to determine the cause. For example, a drop in crime rates throughout the 
county may be due to larger forces such as the improved economy rather than specific interventions 
with at-risk youth. Ideally, with enough agencies working effectively under the auspices of 
Proposition 10, there will be a measurable impact on the whole population or on specific 
communities or groups, and perhaps with the development of improved data capacity over time, 
agencies will develop the ability to link program performance to population results. 6 

Selecting indicators 

An indicator is a measurement that can be used to track changes in conditions over time. “Generally, 
an indicator focuses on a small, manageable, and telling piece of a system to give people a sense of 
the bigger picture.” (Norris et al. 1997). For example, a frequently used indicator is the infant 
mortality rate, which is commonly thought to reflect not only the health of infants, but also the 
overall social health of a population. The purpose of using indicators in the results accountability 
framework is to keep the focus on and assess progress towards desired results. 

Indicator sets will always be imperfect. Those involved in producing children’s “scorecards” have 
commented on the conceptual difficulties and competing values that must be accommodated when 
identifying a short list of indicators intended to convey the complex conditions of children and 
families. There is increasing agreement that the process of selecting results for accountability 
purposes must have political legitimation and must involve those responsible for achieving the 
results (Schorr 1995). One of the fundamental principles of Proposition 10 is inclusion and public 
input in every step of the planning and implementation process. If the results and indicator selection 



6 Adapted from Sutherland C, Shulman E, Halfon N, Gardner S, McCroskey J, From Resources to Results for 
California 's Children and Families: A Guide to Proposition 10 Strategic Planning and Implementation, UCLA Center 
for Healthier Children, Families and Communities, 2000. 



process is to be regarded as credible, useful and reflective of the community’s priorities, it will have 
to include the input of those responsible for achieving the results, namely, service providers, as well 
as those who would be the recipients of programs and services, namely, parents, special-needs 
families and other traditionally underrepresented groups. Such an inclusive process will necessarily 
involve compromises, and could potentially either broaden or bias the story told by the indicators. 

The Proposition 10 State Commission Results Document suggests pursuing four long-term strategic 
results: 

> Improved Family Functioning: Strong Families 

> Improved Child Development: Children Learning and Ready for School 

> Improved Child Health: Healthy Children 

> Improved Systems for Families: Integrated, Comprehensive, Inclusive and Culturally and 
Linguistically Appropriate Services 

For each of these long-term results, a set of short-term results can be developed which lay the 
groundwork for achieving the longer-range goals. The Results document focuses on possible 
shorter-term results, which emphasize building the capacity of the county to provide effective 
supports and services to families with young children (process indicators) to support the ultimate 
achievement of long-term results. (Examples of these indicators can be found in Appendices B and 
E.) 

Using a conceptual model for indicator selection 

Using a conceptual model to guide the indicator selection process helps to clarify why certain 
indicators are selected and others are not. Since indicators are often used to decide where resources 
should be directed, indicator selection is as much a political as a technical process (Young, Gardner 
& Coley, 1994) and will be driven by both the values of the stakeholders and the research evidence 
supporting particular issues. It is helpful if the values of participants in the selection process are 
openly discussed as part of a consensus-building process. 

A well-defined conceptual framework shows the linkages between the indicators and provides a 
context which gives meaning to child health and well-being outcomes. An example of such a 
framework is the Lifecourse Health Development Model (Halfon and Hochstein 2001). This model 
builds on the “Field Model” of the multiple determinants of health (Evans & Stoddart 1990) by 
adding to it a lifecourse perspective. 

The Field Model 7 is holistic in that it includes and shows the interconnections between a broad range 
of factors (e.g., genetic factors, physical and social environments, health care, level of prosperity, 
and individual behavior and biology) which contribute to a child’s physical, mental and social 
functioning, and overall well-being. (See Appendix H.) For example, a child’s physical 



7 This description of the Field Model is adapted from “Determinants of Children’s Health and Development”, L.A. Health 
(Jan 2000). P Simon, C Wold, J Fielding, A Long (eds.), Office of Health Assessment and Epidemiology, Los Angeles 
County Department of Health Services, Public Health. 
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environment includes the safety of his/her home, child care site, school, and neighborhood, including 
exposure to toxic substances, violence, injury or infectious diseases. A child’s social environment 
includes the quality of interactions and functioning of individuals and groups in the child’s family, 
community and society at large. Social factors that can affect a child’s health and development 
include poverty, family functioning, the educational system, social networks and the level of violence 
both in the home and in the community. A child’s genetic endowment may be directly linked to a 
negative health outcome (e.g., congenital heart disease or oral cleft) or it may increase the risk of 
acquiring certain conditions later in life (e.g., developing a chronic illness such as alcoholism or 
schizophrenia). The value of the Field Model is that it emphasizes the broad range of determinants 
and multiple factors that influence a child’s health, suggesting that a cross-disciplinary, multi-faceted 
approach should be considered when selecting indicators and developing strategies to improve 
children’s lives. 

Halfon and Hochstein (2001) have expanded on the Field Model by adding the idea of 
developmental “trajectories,” or pathways that individual development follows over the course of 
a lifetime (see Appendix I). Individual trajectories are influenced by different initial endowments, 
as well as by risk factors and determinants that are encountered throughout the lifecourse. Protective 
factors such as health-promoting behaviors, healthy environments, and specific interventions, serve 
to improve overall trajectories. Risk factors such as poverty, exposure to or use of alcohol, drugs 
or tobacco, and violent home environments often result in lower overall functioning and health, and 
possibly accelerated declines later in life. The timing, content, and synergy of health promotion, 
disease prevention and developmentally appropriate interventions can have a meaningful impact on 
the long-term effects of those risk factors. This can be especially true during “critical” or “sensitive” 
periods, such as early childhood and adolescence. The recent focus on the importance of the very 
early years (0-3) for brain development, which sets the stage for all later development, points to this 
time as a critical period where successful interventions could have a significant impact on the future 
lifecourse trajectory of individuals. 

According to the Lifecourse Health Development Model, different determinants of health and well- 
being will have different magnitudes of effect at different stages of the lifecourse. For example, 
family factors appear to be the predominant influence on the health and well-being of young 
children. As individuals get older, the role of individual behavior and lifestyle has a greater impact 
on their health and well-being. 

By using a conceptual framework such as the Lifecourse Health Development Model, indicators are 
selected and presented in such a way that they tell a coherent story about the impact on children’s 
well-being of multiple factors at different stages of the lifecourse. By highlighting the importance 
of experiences in the very early years for later outcomes, this model could provide a rationale, for 
example, for including indicators that capture the importance of the parent/caregiver-child 
interactions in the critical 0-3 years period, or that assess school-readiness at different developmental 
stages such as at 3 years and again at 5 years, in addition to the commonly used third grade reading 
scores. This would reflect the importance of monitoring what is going on earlier in the 
developmental course and assessing the impact of both risk and protective factors on school- 
readiness. 
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Another useful conceptual framework is that of “critical pathways” (Halfon et al. 2000). A critical 
pathway 8 is a mapping of the significant factors and the relationships between them that influence 
a specific outcome. Pathways are generally mapped in a causal sequence starting with social and 
economic factors, such as education and poverty, and progressing to individual behaviors related to 
the outcome, such as smoking or the utilization of prenatal care. The following shows a critical 
pathway leading to the outcome of low-birthweight babies. It includes the following determinants: 
poverty, public safety, access to transportation, and utilization of prenatal services. 







Critical Pathway 






Economic & Social Factors 




Health Behaviors 




Health Outcomes 


Poverty 

Public Safety 

Access to Transportation 




Utilization of Prenatal 
Services 




Low birthweight 



Meta-determinants are those key factors that play a significant role in multiple health outcomes. 
Halfon et al. (2000) found that the following meta-determinants had the greatest impact on child 
well-being in California: 



> poverty/socio-economic status 

> maternal substance abuse 

> low level of maternal education 

> family breakdown/unmarried status 

> maternal depression 

> lack of prenatal care 

> child neglect 

> lack of access to medical care 



Mapping determinants of child outcomes in a critical pathway and identifying meta-determinants 
help to identify the most strategic leverage points for the allocation of Proposition 10 funds and 
provides a strong rationale for selecting indicators, strategies, and performance measures to track 
their results. 



8 This description of critical pathways is adapted from “Determinants of Children’s Health and Development, ” L.A. 
Health (Jan. 2000). P Simon, C Wold, J Fielding, A Long (eds.), Office of Health Assessment and Epidemiology, Los 
Angeles County Department of Health Services, Public Health 



Criteria for indicator selection 



Using a set of criteria to guide the indicator selection process also provides a strong rationale for 
including specific indicators. We suggest the following criteria for selecting individual indicators, 
which are presented in the order of their significance: 

1 . Importance 

2. Meaningfulness 

3. Validity and reliability 

4. Sensitivity to change 

5. Availability and quality of data 

6. Ability to induce action 

7. Facilitates cross-systems collaboration on behalf of children 

1. Importance: How important is this indicator as a component or determinant of child and 
family well-being? 

An indicator is considered important to the degree that it reflects a significant contribution to the 
health and well-being of children and families. An indicator can be important in terms of: 

> Public policy — It has the potential to engage policymakers or address an area 
of policy interest (e.g., Proposition 10). 

> Epidemiologic principles — It represents a significant attributable risk; that is, 
it accounts for a significant proportion of an undesirable result in the 
population (e.g., tobacco use accounts for 19% of all deaths). 

> Community values and priorities — It reflects values important to the 
community and has the potential to facilitate relationship-building and foster 
buy-in among diverse community sectors (e.g., school-readiness is a result of 
many different factors present in early childhood, for example, family literacy, 
quality of child care, health care, safe home environment, etc., and requires the 
involvement of multiple stakeholders). 

2. Meaningfulness: Can the public and policymakers easily understand what this indicator means 
and what its implications are for child and family well-being? 

An indicator is meaningful if it reflects a common sense understanding shared across groups 
(including the general public, policymakers and service providers) and most people would think 
it was important. An indicator is likely to be more familiar and meaningful if it appears 
frequently in the media or other health reports. 
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3. Validity and reliability: Does this indicator measure what it is supposed to measure? 

An indicator is valid, or scientifically sound, if it actually measures what it is intended to 
measure. A reliable indicator is one for which repeated measurements from a population produce 
similar results; it can be used reliably across diverse population groups; and data are compiled 
using a systematic and fair method with consistency each year. 

4. Sensitivity to change: Is this indicator sensitive enough to pick up and respond to the impact of 
interventions? 

An indicator is sensitive to change if it responds relatively quickly and noticeably to interventions 
made so that measurement over time has the potential of reflecting the impact of behavioral, 
policy or procedural change. 

5. Availability and quality of data: Are data readily and consistently available for this indicator? 

Once results are identified, there are numerous technical problems related to the availability, 
reliability and validity of key data elements, including whether: 

> Data are collected regularly and routinely. 

> Data are readily available from established sources on a regular (e.g., 
annual) basis. 

> Historical data are available to provide trend data. 

> If data are not compiled in one place, the information can be readily 
collected and compiled on an annual basis. 

> The cost, time and effort of collecting new data necessary to construct the 
measure are feasible. 

> The costs involved in purchasing or analyzing the data are reasonable. 

> Data are available at the appropriate level of geographic detail. 

> Variables necessary for risk adjustment are available in the data set. 

> Data are of high quality, with few regular systematic reporting problems, 
such as misreporting, non-representativeness of the sample, or under- 
reporting 

6. Ability to induce action: Will this indicator grab the attention of the public and policymakers 
and help motivate action? 

Indicators are more likely to be translated into action if they grab attention, increase awareness, provide 
information or knowledge, engage or motivate, and link to action at the individual or population level. 

7. Facilitates cross-systems collaboration on behalf of children: Does this indicator reflect an 
issue that requires the involvement of stakeholders from different sectors of the county? 

Indicators that reflect conditions that clearly require the efforts of multiple departments, agencies 
and community groups are more likely to galvanize collaborative efforts and to inspire service 
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providers and community groups to action. Since the well-being of young children and their 
families clearly requires the efforts of many different partners, it is important that indicators 
signal the need for ongoing collaboration. 

Finally, commissions can also use a set of criteria for the indicator set as a whole (as opposed to the 
individual indicators in the set). We suggest the following criteria: 

1. Cohesiveness: The set of indicators is consistent with and reflects the underlying conceptual 
framework being used. Each indicator would reflect an aspect of the framework. 

2. Interpretability: The set can be made meaningful to the target audience. For example, it creates 
“pivot points” 9 for community engagement; it generates stories about child and family well-being 
that help the data resonate with a lay audience of community members, including parents; it 
develops key messages and makes the issues it represents meaningful in terms of the lifecourse 
of individuals, their own assessment of risk, and the changes they might be willing to make in 
their own lives and be willing to support in the life of their community. 

3. Capacity to facilitate collaborations and stimulate partnerships: The set stimulates 

partnerships and collaborations among community leaders and organizations involved in various 
aspects of children’s health and well-being: health care providers, child care providers, social 
service agencies, schools, businesses, community-based organizations, the faith community, law 
enforcement, parents, etc. In order to engage a cross-section of people, the set would need to be 
innovative and inclusive and capture a vision of child and family well-being that would engender 
buy-in from diverse perspectives. 

4. Ability to improve management accountability and community health improvement. The 

set can support program management and accountability as well as community education. A key 
role of the set would be to stimulate initiatives at the community level to support school- 
readiness, to monitor the effect of these initiatives over time, and to serve as the organizing 
framework for communicating results of these initiatives to the community on a regular basis. 

Sources of indicators 

When selecting indicators, the commission may want to consult other reports and organizations 
already tracking information for young children and families. These might include: 

> CCFC County Results and Indicators Data Collection Tool 

> State Commission Results Document 

> Annie E. Casey Foundation KIDS COUNT reports 

> Children Now annual Report Card 

> Foundation Consortium, Quality of Life Indicators of Children & Families 

> Center for the Study of Social Policy, Improved Outcomes for Children Project 



9 



Issues that inspire collaborative action by a diverse group of people. 
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> America ’s Children: Key National Indicators of Well-Being, 1999 

> Trends in the Well-Being of America ’s Children and Youth, 1 998 

> Completed Proposition 10 county strategic plans 

> Local, state and national children’s reports and scorecards 

> Local county surveys on children and families, 

> Local county children’s planning collaboratives 

> United Way 

> The Search Institute 

(See Appendix G for contact information.) 

Traditional and new measures of health and development 

Appendix C presents a matrix of indicators of child health and well-being used by some of the most 
well-known indicator reports. While commissions can make a good start by relying on available 
data, there is also a need for new kinds of indicators and data collection that more accurately reflect 
the conditions of children and their families. Appendix D gives some examples of the kinds of 
additional data that would be useful for tracking Proposition 10 results. 

Current data systems were designed primarily to monitor compliance with federal, state and local 
regulations and to assure funders that resources were being used well. They focus on the negative 
conditions that led families to seek help rather than on the positive aspects of their lives, the “well- 
being” of parents and their children. Although some population-based data are also available 
through the census and special surveys, there is too little information on “normal” conditions, daily 
concerns and the realities of life in neighborhoods across the state. There has been increasing 
interest in developing new indicators to measure the assets and capacities of children, families and 
communities, in addition to needs, to present a more balanced picture of children’s well-being and 
to allow programs and strategies to build on existing strengths and assets (Ben-Arieh 2000; Search 
Institute). 

Certain indicators can be considered “sentinel” (i.e., warning signs of certain conditions not being 
in place to support optimum health and/or well-being) and “nodal” (they reflect a combination of 
circumstances that are important for a positive result). For example, in order to have long 
breastfeeding duration, the mother has to initiate breastfeeding and be supported in continuing to 
breastfeed. This requires that a variety of processes and supports be in place, including birth-hospital 
support, family support, as well as information and support from home and well-baby visits, WIC 
centers, the workplace, etc. Therefore, breastfeeding duration is a very important sentinel indicator 
of whether these processes and supports are working together in an effective and efficient way. It 
is nodal in the sense that it means the hospital system, the doctor’s office, home visiting services, 
the WIC Centers and employers must all have been doing their jobs right in order to produce a 
positive result. In other words, many different factors have to come together and work well if that 
indicator is going to improve. 
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School-Readiness Indicators: Given the focus on school-readiness as an important outcome by the 
State Children and Families Commission, counties might consider the implementation of a universal 
kindergarten assessment in order to measure school-readiness. While the term “school-readiness” 
has a complex past, and a checkered recent history, new formulations of this outcome and new 
measures may lend themselves to easier and more accepted use. For example, the state of Maryland, 
as part of an early childhood initiative, will be using the Work Sampling System, developed by Sam 
Meisels of the University of Michigan, to assess school-readiness among Maryland kindergartners. 
If schools in California were to institute a kindergarten school-readiness assessment, it would permit 
kindergarten teachers to have a sense of the capacity and readiness to learn of entering students, and 
would also serve as a very useful tool to monitor the impact and effectiveness of Proposition 10. At 
present, the only universal assessment measure that is used in California is the Stanford Nine Test 
given in third grade. This type of measurement, while useful, is coming 3 to 4 years after the 
interventions. 

It might also be important to consider how a universal assessment could be carried out for all 3 -year- 
olds. With the growing move towards universal preschool education, some type of preschool- 
readiness assessment might also be instituted on a voluntary basis, but then regularized in some way. 
Such a preschool-readiness assessment might focus more on social-emotional issues and language 
development as important precursors of eventual school-readiness. 

System Development: In addition to indicators of child and family development and well-being, it 
will also be important to track indicators of system development, coordination, collaboration and 
integration. If the system were more integrated or moving toward greater integration, how would 
we know? Appendix E provides a list of suggested indicators of service integration, accessibility 
and cultural appropriateness listed in the State Commission Results Document. 
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IV. Issues in Integrating and Coordinating Data Collection 

Uses of outcomes and indicators for planning and monitoring 

Most people who provide services for children and families are used to using data after programs 
have been implemented to evaluate their success. The results accountability approach requires that 
Proposition 10 commissioners and planners begin the planning process by defining desired results 
and indicators, then shaping programmatic strategies to achieve those desired results. The same 
results and indicators will then be used to provide a framework for program monitoring, guiding the 
definition of performance measures against which program results will be judged. Data from 
program monitoring can be used to improve and fine-tune programs, data on countywide progress 
toward desired results can be used to guide future funding decisions, and both kinds of data should 
be continuously fed back into ongoing planning. The following illustrates the kinds of activities 
needed to integrate and coordinate data functions to achieve both planning and monitoring purposes. 



1. Proposition 10 Commission agrees on desired results and defines the indicators that will 
be used to measure progress toward these results. 



Example 

Result: Improve health of newborns 

Possible Indicators: Birthweight, neonatal exposure to alcohol, tobacco and other drugs 



2. A “data committee” is convened to define and oversee key data collection and monitoring 
functions. 



Key data collection and monitoring functions would include: 

> operationalizing (deciding how to measure) indicators 

> coordinating with the many groups that collect relevant data 

> reviewing data reports to determine their significance for Proposition JO 

> analyzing trends and making projections 

> providing technical assistance to commissioners and workgroups 

> working with staff/consultants 
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3. Staff, consultants and data committee members work with planning group(s) to educate 
them about issues, terminology and data requirements, helping them to become 
comfortable with using data to inform planning. (The designated leaders need to take a 
hands-on approach because many commissioners will be uncomfortable working with data, 
preferring to see this as a “technical” function that is outside their experience or sphere of 
influence.) 



Activities might include: 

> training sessions for commissioners and interested community participants on using data for planning 

> meetings with work groups to discuss the specific data issues associated with key topic areas 

> using examples to work through the process of moving from results to indicators to performance measures 



4 . Staff and consultants review the research literature for promising strategies, best practices 
and data on program effectiveness that may be helpful in planning. They also solicit 
reports from local service providers (who may not have written up their findings in a 
formal manner) on effective strategies. 



Example 

In Los Angeles County the Department of Mental Health commissioned a panel of experts from UCLA to search 
the literature for program strategies that have been successful in reducing the mental health problems of young 
children and their families. Other reports in this series provide similar information. 



5. Staff, consultants and data committee members work together with local service providers 
and service recipients to design a core set of performance measures that will be used to 
assure program quality, monitor accountability and track success for each major 
programmatic strategy pursued (including system integration). The “core set” of 
measures should be clear, practical, applicable across programs and not unduly time- 
consuming for service providers. These performance measures would be used for 
individual programs and across allied programs (i.e., those that address the same 
indicator). 
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Example 

Indicator : Access to health care 

" What works ” strategy: Community outreach to increase enrollment in Medi-Cal, Healthy Families and participation 
in the CHDP (Child Health and Disability Prevention) 

Possible performance measures for community outreach strategy: 




Quantity 10 


Quality 


Input 

(Process) 


# of persons contacted 

# in each region/city 
demographic characteristics 


% eligible but not enrolled in Medi-Cal and 
Healthy Families 
% participating in CHDP 


Output 

(End Result) 


# of persons enrolled/participating 


% enrolled who regularly see a health provider 
% participating in CHDP who received services 
determined to be needed 



10 For further explanation of this format for presenting performance measures, see Friedman, M. Results Accountability 
for Proposition 10 Commissions: A Planning Guide for Improving the Well-Being of Young Children and Their 
Families , in N Halfon, E Shulman, M Hochstein and M Shannon, eds., Building Community Systems for Young 
Children, UCLA Center for Healthier Children, Families and Communities, 2000. 
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When developing performance measures, there are several points to keep in mind: 11 

> It is important to clarify the target population, service delivery boundaries and strategies of 
the program or agency being measured. 

> Both process/capacity measures and end results are important. Process measures are useful 
in the early stages of developing a results accountability system when there has not been time 
to achieve end results; more end-result measures will be added over time. Process measures 
are also important throughout the process when linked to specific results. The linkage 
between process measures and end results needs to be continuously measured, since it is only 
through understanding how the process is related to an end result that communities and 
service providers know on which lever they can pull in order to achieve their desired results. 
Since most results have many steps along the pathway and several different processes that 
can lead to the desired result, it is only by continuously measuring the process measures that 
one understands where and how the change is taking place and can make adjustments. This 
knowledge serves as the basis for continuous quality improvement systems. 

> In the table shown above, the most important measure is “customer/client” results in the 
lower right quadrant (e.g., improvements in skills, attitudes, behavior, circumstances). 

> There should be a method at the county level to oversee and be responsible for results 
accountability across programs and systems so that similar programs are measuring similar 
results. 

> In order to track outcomes, most agency programs need to go beyond their currently available 
data, using, for example, customer/client surveys and trained observer ratings. Agencies will 
vary in their capacity for data collection. 

> Data should be collected, analyzed and reported internally at least on a quarterly basis. 

> Breakouts by geography, client demographic characteristics, organizational units providing 
the same service, by difficulty of the incoming workload, and by type and magnitude of 
service enable users to identify where and under what conditions programs and services 
appear to be more successful or less successful. 

> Results need to be compared to other data, e.g., the previous year’s performance or other 
benchmarks. 

> Appropriate explanatory information helps to clarify the performance data. 

> All levels of agency personnel should be trained in a results orientation, not just upper-level 
officials and/or managers. 

Compared to program evaluation, the performance monitoring approach has both benefits and 
drawbacks. The primary benefits are: 



1 1 This section was developed from the following sources: Hatty, HP. Performance Measurement: Getting Results. 
The Urban Institute Press, Washington, D.C, 1999 and Friedman, M. Results Accountability for Proposition 10 
Commissions: A Planning Guide for Improving the Well-Being of Young Children and Their Families, in N Halfon, 
E Shulman, M Hochstein and M Shannon, eds., Building Community Systems for Young Children, UCLA Center for 
Healthier Children, Families and Communities, 2000. 
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> Performance monitoring builds on a framework of indicators that are clearly defined and 
spelled out in advance of program implementation. 

> Performance measures are standardized, allowing for easy comparison across programs. 

> Standardized approaches may save money by not requiring a new evaluator and evaluation 
plan for each program. 

The primary drawback is that, without careful analysis of the context of each program, performance 
data do not, by themselves, explain why and how the outcomes occurred. If program participants 
improve, is their improvement the result of the program, other factors, or some combination of the 
two? Although “gross” improvements may have occurred, what percentage of improvement can be 
attributed to the “net” effects of the intervention? Given the complexity of program evaluation, 
however, many experts believe that it makes more sense to invest in comprehensive performance 
monitoring approaches for all programs. In this approach, more intensive program evaluation designs 
would be reserved for well-established programs whose staff feels comfortable with program design 
and operations, and for which there is some reasonable evidence of success from tracking 
performance. 

6. Staff, consultants and data committee members identify key sources of available data 
countywide, developing partnerships with the agencies and organizations that collect and 
maintain these data sources (including adding representatives to the data committee). It 
is important to understand that maintaining complete data on children, families and 
communities does not fall under the jurisdiction of any one organization - rather there are 
many different kinds of organizations that collect data relevant to commission purposes 
(also the Commission’s definition of “relevance” will change over time). Don’t assume that 
the Commission has to collect and maintain all of the data it will need; assume, rather, that 
it will develop a broad range of partnerships that will enable it to find data when needed. 



Key data sources might include: 

> County departments that serve young children and their families, such as 
public social services, health, mental health, probation, courts, district attorney 

> City departments including parks and recreation, libraries, social service departments, and child care 
coordinators 

> School districts and County Offices of Education 

> Child care resource and referral agencies 
y United Way 

y Information and referral agencies 
y Head Start programs 



Some new sources of data will undoubtedly also be needed to augment available data using 
techniques such as questionnaires, surveys, focus groups and parent interviews. Thus it will be 
desirable for county commissions to commit a portion of their Proposition 10 resources toward 
building data capacity within their counties. Smaller counties can partner regionally with other small 
counties or with a larger county nearby to facilitate data collection and sharing. 
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7. Identify a team of researchers/evaluators to develop appropriate evaluation methodologies 
and to analyze data. This may include commission staff as well as contracted evaluators. 

Part of their responsibility should be to provide ongoing feedback to program operators 
to fine-tune and improve programs. 

8. Provide annual reports to decision-makers and the public. This could be in the form of a 
countywide scorecard tracking the conditions of young children and their families. Ideally 
there should be several versions of the scorecard for different geographic regions of the 
county and/or for different ethnic groups. 

Advocates have pointed out some of the difficulties of using data to develop action strategies to 
improve the conditions of children and families. One problem is that some people do not understand 
or are not comfortable with data in a spreadsheet format. They may not know how to read or 
interpret the data, or it may be presented at an aggregate level (for example, statewide or countywide) 
when they want to take action in their own communities, or the indicators may not point directly to 
practical action. For example, figuring out strategies to decrease the number of families who live on 
incomes below the poverty level can be overwhelming to the even the most dedicated activists. 
Other kinds of indicators — such as the number of children who have access to free or reduced price 
breakfasts at school — may make more sense for a local action agenda. 

Countywide report cards can be a useful tool for presenting data in a format that can educate and 
inform the broader public as well as policymakers in a more user-friendly way. When designed well, 
report cards tell the stories behind the indicators, including the reasons why the findings are what 
they are, the direction in which trends are heading, how the indicators are interrelated, and the 
implications of the trends for the county as a whole, as well as for specific subpopulations. Report 
cards can be designed in an attractive, accessible format presenting information in multiple ways 
(e.g., text, pie charts, line graphs, quotations, photographs), while avoiding overwhelming the reader 
with numbers and tables. By providing interpretive text and linking the indicator information to 
existing and potential action in the community, report cards can help to create the bridge between 
data and action. The Proposition 10 Commission can collaborate with other countywide data efforts 
to produce a report that demonstrates the linkages of Proposition 10 activities to others related to 
children and families. 

In response to the expressed need for user-friendly data on smaller areas (smaller than states, 
counties or cities), several national foundations are focusing on the development of community- or 
neighborhood-level data that could be made widely available to community-based groups. For 
example, the National Neighborhood Indicators Partnership is a collaborative effort between the 
Urban Institute in Washington, DC, and local partners around the country to further the development 
and use of neighborhood-level information systems in local policy making and community building. 
All local partners in this initiative have, or intend to build, locally self-sustaining advanced 
information systems with integrated and recurrently updated information on neighborhood conditions 
in their cities. These information systems are designed to facilitate the direct use of information by 
local government and community leaders in order to build the capacities of distressed urban 
neighborhoods. (See Appendix G: Websites and Online Resources.) 
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9. Feed findings on program effectiveness back into the planning process so that the 
commission becomes a “learning organization,” one that builds on its successes, 
acknowledges its mistakes, moves on, and strives to refine its vision and improve its 
operations. 

One of the most important aspects of the results-based approach is assuring that all of those involved 
in planning receive regular feedback on how the projects are implemented, what works and what 
doesn’t and key lessons learned in the process. This means that someone (or many people) will need 
to spend time translating the ’’data” (statistics, numbers, charts) into “information” that is useful and 
accessible to a broad range of people. It is easy for researchers who have become familiar with the 
intricacies of spreadsheets, tables and graphs to assume that everyone sees the same things they do 
in an array of numbers. Many people are intimidated by numbers, don’t know who to interpret “raw” 
data, or can’t make the leap from specific findings to their general implications for Proposition 10 
planning. 

Proposition 10 Commissions that value informed community input in the planning process, will 
make the extra effort to assure that findings are interpreted for several different kinds of audiences 
(i.e., parents, service providers, policymakers). For example, in England the Department of Health, 
the primary sponsor of research on health and well-being, requires researchers to produce a number 
of versions of their reports directed to different audiences concerned with the issue (Department of 
Health 1995). 

10. Use data to “tell the story” of Proposition 10. What results are the Commissions seeking? 
What has been achieved? What has been learned? 

One way to use findings of results-based strategies is to “tell a story” about what happened and why 
it happened that way. This requires stepping back and giving a broad overview. Rather than writing 
a typical research report, authors should try to describe what the Proposition 10 Commission had in 
mind in funding the strategies it did, what happened as a result of implementing those strategies and 
what was learned in the process, in terms understandable to a lay audience. The notion of telling 
a story goes further than just cutting out the jargon or simplifying the research findings, it requires 
that the authors keep in mind the essential elements of a good story - that it has a storyline, 
emotional appeal and conveys the essence of what happened. 

Using existing information systems 

Many of the information systems already in place in each county collect data relevant to Proposition 
10. Although most were designed for reporting to funding sources, not for planning, these data can 
be useful if ways can be found to integrate or at least to analyze data across systems. Integrating data 
on children and families from many different systems into one integrated database has been a dream 
in many localities, but there are so many difficulties that it has seldom been accomplished. These 
challenges include both technical and political hurdles): 
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“The technical issues are probably all too familiar: stove piped legacy data 
systems, none of which easily communicate with each other, and each employing 
just enough idiosyncrasies of attribute definition as to confound any but the most 
fearless (or ignorant). 

The “political” issues were those — also familiar, I suspect — of turf-protection 
and general suspicion that, in the wrong hands, “our” data will be “mis-used” or, 
at best, “misunderstood.” (Murphey 1999: 77) 

Commissions in most counties will face at least some of the following technical issues especially 
endemic to bureaucratic, government-operated information systems: 

> separate systems for different departments or functions that rely on different kinds 
of (usually outdated) hardware and software, guided by different standards and 
criteria; 

> different ways of defining essential data elements, including the unit of focus (family, 
child, household, etc.), geography (census tracts, zip codes, cities, school districts, 
etc.) and services used (visits, days in care, “slots,” etc.); 

> different reporting periods (average daily attendance in schools, monthly welfare 
roles, etc.); 

> inability to track individual children over time or across systems; 

> inability to derive an unduplicated count of individual children served (because the 
system counts things like immunizations given, CHDP visits, etc., rather than the 
children who receive them); 

> difficulty matching data on children served by more than one system; 

> different definitions of data elements which are mandated by state and federal 
government departments (and therefore not under local control); 

> difficulties in negotiating confidentiality protections; and many more. 

Proposition 10 commissions may want to draw on the efforts of existing statewide groups that are 
trying to improve data integration. For example, the Family Outcomes Project has convened a 
number of experts from across the state to improve data collection and integration (see Resources). 
In a project funded by the Stuart Foundation, the Healthy Start Field Office and the California Center 
for Community-School Partnerships at UC Davis have helped to integrate data on school sites for 
school-based Healthy Start projects. 

Because of the many difficulties of matching data across systems, people in some counties have 
investigated the possibility of assigning each person a single identifier that could be used across 
county departments. The most obvious possibility is the Social Security Number, but the potential 
negative impact on privacy of using such a widely recognized number would be considerable. 
Others have suggested that county government departments could construct a process for assigning 
identifiers that could be used across departments (for example, first two letters of first name, first 
four digits of street address, etc.). To our knowledge, little progress has been made in this effort. 
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Even if it is not possible to integrate the many different data systems that could provide useful 
information, it should be possible to develop agreements under which agencies will provide 
comparable data so that commission staff and evaluators can use a standard analytic framework to 
assess data. Commissions should also identify a local research team (from higher education or 
community-based agencies) that can provide knowledge of the research literature and advice about 
data analysis and evaluation methods. 

In terms of the political issues. Proposition 10 commissions will need to understand the local “map” 
of data sources in order to negotiate working partnerships with a broad range of organizations that 
collect and analyze relevant data. Proposition 10 recognizes the interdependence of all of those who 
work with and on behalf of young children and their families, and most commissions will quickly 
realize their own reliance on others for accurate, timely data. 

Creating new data sources 

Since many potentially important data elements are not available. Proposition 10 commissions will 
need to think about creating new data sources or partnering with others to enhance existing systems. 
Methods include: 

> creating new countywide data coalitions; 

> assembling countywide or local population data in one place (e.g., working to identify the 
best available data and to post these data sets on the web); 

> using focus groups and parent interviews; 

> influencing state departments to adapt mandatory data elements in current reporting systems; 

> influencing development of local data systems; 

> creating Proposition 10-specific data systems; and 

> requiring data collection for evaluation purposes in all community programs or in all 
programs in a particular service sector (e.g., child care). 




The Center for Collaboration for Children, California State University at Fullerton 
The Early Care and Education Needs Assessment (ECENA) 

The Center for Collaboration for Children at California State University at Fullerton in Orange County (see Resources) 
is conducting The Early Care and Education Needs Assessment (ECENA), a set of 1 1 linked studies and activities that 
will compile a wider array of information on early care needs in Orange County than has ever been assembled. Funded 
by the Proposition 10 Commission, Head Start of Orange County and prospectively, the Social Services Agency, the 
study seeks to answer the following questions: 

1 . What are the current early childhood care and education options in Orange County? 

2. Which groups of parents utilize which kinds of care, and why? 

3. How do parents and teachers define quality early childhood education? 

4. How do parents and teachers define school-readiness (the characteristics that make a child ready for 
kindergarten)? 

The ECENA uses several different strategies to answer these questions: 

1 . Parent surveys 

2. Provider survey 

3. Provider focus groups 

4. Elementary teacher survey 

5. Elementary teacher focus groups 

6. Service data analysis 

7. Review of prior studies 

The first step in this process is the parent survey. This phone survey is currently being conducted, and addresses the 
following: 

1 . Who are the parents of young children in Orange County? (An assessment of demographic characteristics such 
as age, ethnicity, geographic distribution and education.) 

2. What kind of early care are parents of young children currently using? 

3. Why did parents choose their current care arrangements? 

4. Are parents satisfied with their current care, and why (or why not)? 

5. How do parents define quality early childhood education? 

6. How do parents define school-readiness (the characteristics that make a child ready for kindergarten)? 



Issues of primary data collection will be somewhat different for the smaller, rural counties, where 
the resources available to gather data can be limited. Smaller counties can conduct face-to-face 
outreach using locations frequented by parents such as markets, shopping malls, and schools to 
conduct surveys. In Sierra County, for example, efforts were made to contact every parent in the 
county through such in-person surveys. Using outreach workers from the target communities to 
conduct local surveys can also be an effective method, as was done in Santa Cruz County. 

A source of data that will be available to counties in the near future is the California Health Interview 
Survey (CHIS). CHIS will survey 55,000 households throughout the state, including a sample of 
6,000 children ages 0-4. The survey focuses principally on public health and access to health care. 
There is a special Child Core for CHIS which includes questions in the following areas: child’s 
health status (physical, behavioral, mental, dental); health care (utilization, providers, barriers); 
preventive health behaviors; nutrition; health insurance coverage; child care; and demographics. 
Currently, the Child Questionnaire emphasizes health conditions, health behaviors, health care 
utilization and insurance coverage. As recommended by their technical advisory group, the CHIS 
development team is looking at ways to include more indicators on child development in future 
surveys, which will be even more relevant to Proposition 10 issues. 
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CHIS will generate statewide estimates for California, plus local-level estimates for counties with 
populations of 40,000 or more. Estimates for most counties will include breakouts by age groups and 
ethnic groups, depending on sample size. CHIS has the flexibility to expand sample size to allow for 
more extensive sampling for any given group or geographic area in order to improve the reliability 
of these estimates. CHIS data can be used to: 

> assess and monitor the health of populations, both statewide and locally, including 
prevalence of chronic conditions, communicable disease, social and physical environmental 
health, and quality of life; 

> more accurately measure health insurance coverage and access to health services at the local 
level and for smaller population groups statewide, including the effectiveness of public 
programs, such as Medi-Cal and Healthy Families, in meeting their needs; 

> assess the health and development of children, both statewide and locally, including social 
and physical development, parent-child relationships, prevalence and management of chronic 
conditions, access to preventive and other health services; 

> identify underserved populations, specific health problems and the impact of welfare reform 
and managed care; and inform and support funding opportunities for special populations and 
local jurisdictions; and 

> assess community needs as required of Medi-Cal managed care contractors, hospitals serving 
Medi-Cal beneficiaries, and all nonprofit hospitals. 

CHIS will operate in 2-year cycles, with data collected every second year. The first 24-month cycle 
began July 1, 1999, and continued through June 30, 2001. Data collection is expected to begin July 
2000, be completed 9 months later, and be ready for release in summer 2001. 

No one “owns” or can “control” all of the data that will be needed to fully understand the impact of 
Proposition 10. It is also important to understand other major initiatives that affect the lives of 
young children and their families that may overlap with Proposition 10 (especially welfare reform, 
other tobacco taxes, state surplus funds). Most counties will not have the data infrastructure in place 
to monitor the impact of Proposition 10, and will need to invest a portion of their funds to support 
such data activities. 

Developing a data infrastructure 

Many county commissions will want to develop or join existing data-oriented partnerships that can 
provide an ongoing infrastructure for cross-organizational data sharing. This “infrastructure” should 
focus on the people and groups who have regular access to data and enhanced data-analysis skills, 
and on those who can help to turn reams of data into information that is useful both to commissions 
and to the many other groups concerned about children and families. 

Commissions should plan to reserve a portion of their funds to invest in the development of such a 
collaborative data infrastructure to inform planning and monitoring of funded services and to ensure 
that Proposition 10 dollars are being allocated in the most effective and efficient way. Such a 
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collaborative and integrated data infrastructure provides the opportunity for adequate monitoring of 
program and population-level results, which in turn supports the development of comprehensive, 
integrated systems of care. 

In Los Angeles County, the Board of Supervisors recently approved the establishment of an 
Integrated Services Branch in the Chief Administrative Office (CAO), which will include a Strategic 
Planning Data Center (SPDC). The CAO and the Children’s Planning Council will be joint leaders 
of the SPDC to promote sharing and analysis of data across county departments, as well as to 
develop partnerships with institutions of higher education and community-based organizations 
concerned with data. The two primary goals of the SPDC are to enable policy decisions and plans 
to be soundly based on data and to enhance tracking of program effectiveness. The State 
Commission can help by working to negotiate similar partnerships between the state government 
departments whose services most affect young children and their families. A reasonable estimate 
of the amount of money that should be invested in research, including collection of county wide trend 
data as well as performance measurement, would be 5-10% of annual funds. This is the range 
allotted to research and evaluation by most of the large federally funded demonstration projects. In 
addition, commissions should plan to invest up to another 5% in the first year or two to develop an 
adequate data infrastructure to support results-based planning and accountability. 
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Alameda County Children and Families Commission 
ECChange 

Alameda County was the first county in the state to appoint a Children and Families Commission, to develop a strategic 
plan and to begin implementing that plan. The data and monitoring system now being developed in Alameda County 
is called ECChange. ECChange is a web-based, integrated, cross-agency, secure information system designed 
specifically for the Alameda County Children and Families Commission “Every Child Counts” (ECC) program. 
ECChange enables the unique identification, tracking and monitoring of families receiving ECC services and informs 
the accountability and evaluation components of the ECC. The system is a secure network that promotes accessibility 
to and exchange of information and services from any entry point into the system and respects the privacy and 
confidentiality rights of children birth to 5 years and their families. The requirements of the system were developed by 
cross-agency workgroups. ECChange assists the commission by performing the following functions: 

• Registers and uniquely identifies children, families, and early childhood educators receiving ECC services. 

• Maintains a family/household record that includes children, primary caregivers and other household members. 

• Collects core data elements such as birth name and date as well as confirmatory data elements such as Social 
Security Number and a mother’s maiden name. Additional data are collected for case management, outcome 
indicator reporting, billing, or data needed to meet funding requirements. 

• Provides a mechanism to trigger the deployment of Public Health Nurse Home Visits within 48-72 hours after 
a newborn is discharged from the hospital. 

• Enables all agencies providing home-visiting services to perform data entry during home visits by using hand- 
held devices or laptops. 

• Shares data with Alameda County agency systems that are participating in Every Child Counts. 

• Provides standardized and ad hoc reporting programs from which data can be exported into statistical packages 
such as SAS and SPSS. 

• Protects client confidentiality through strict security protocols which control access that is appropriate to 
agency and user type, and documents client’s informed consent to share information. 

• Automates data exchange between databases: vital statistics (AVSS), Public Health Nursing System (NRS), 
Social Services data systems (CWSCMS and CDS), Adolescent Family Life Program (Lodestar), the state 
Common Application Transaction System (CATS), as well as other databases. 

The ECChange information system can be accessed via the World Wide Web using established Internet standards. 

In addition, it provides sophisticated data analysis and a robust security framework. 



There are several software packages developed by private vendors that are currently available for 
purchase by county commissions to assist them with program management, evaluation efforts and 
other aspects of their operations. These applications have the capacity to capture program-level 
information required for county- and state-level evaluation reporting. The document. Program 
Management/Evaluation Support Systems for County Children and Families Commissions: An 
Assessment of Technology and Service Options 12 analyzes software systems developed by three 
vendors: Advanced Business Software (ABS), Corporation for Standards and Outcomes (CS & O) 
and Pran Infotech, Inc. The analysis discusses the strengths and drawbacks of each system and is 
designed to provide county commissions with the information they need to make an educated 
decision on which software product or technological approach would best meet their program 
management and evaluation support needs. 



12 Published by the Proposition 10 Technical Assistance Service Center (TASC). (See Appendix G: Publications.) 
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The Corporation for Standards and Outcomes 
Outcomes Collection, Evaluation and Reporting Services (OCERS) 

The Corporation for Standards and Outcomes (CS&O) has developed a comprehensive approach for managing the design, 
implementation and maintenance of an Outcomes Collection, Evaluation and Reporting Services (OCERS) system for 
California Children and Families Commissions. OCERS is a combination of services provided by CS&O that will enable 
county commissions to collect, analyze and report countywide data, trends and program outcomes through a central 
Internet application. OCERS provides commissions with the capability to report on the long-term outcomes achieved 
for each child served and each project funded, along with short-term milestone goals and achievements toward fulfilling 
project objectives and indicators. OCERS answers such questions as: 

• To what extent did the commission achieve its goals as defined in the Strategic Plan? 

• What are the nature and extent of the services provided by commission grantees and the extent to which they 
achieved their individual evaluation objectives? 

• How intensive were the presenting health and human service needs of the children and families served by the 
commission’s programs? 

• What short- and long-term outcomes were achieved, and which service delivery models and processes had the 
most impact? 

• How were the funds allocated across outcome domains specified in the commission’s Strategic Plan, and how 
much did the achieved outcomes cost? 

The software package is supported by an experienced team of evaluation experts who provide ongoing and intensive 
technical assistance. The Orange County Children and Families Commission began working with CS&O in April 2000 
to design a comprehensive outcomes collection and reporting service to manage its Proposition 10 funds. In February 
2001, CS&O began working with Contra Costa County and in June 2001 with Kern County to track and report its 
outcomes and contract information. 

This type of software system has the potential to offer tremendous support to many county commissions. However, the 
cost of such a system may be out of reach for many commissions. It is best suited to the budgets of larger counties, or to 
a collaborative group of counties with smaller budgets. 



Telling the story 

Ultimately, data should help the commission to “tell the story” of Proposition 10 to many different 
constituencies with different perspectives, values, assumptions and attitudes. The set of outcomes 
and indicators selected should provide a clear roadmap of the priorities of the commission, its goals 
and its values. Data-based reports should demonstrate how the commission is approaching its 
planning tasks, why it has chosen its priorities, and ultimately how effective its strategies have been. 

Data can increase the quotient of rationality in planning, policy development and 
resource allocation decisions, whether at the program, agency or service system 
level. Data help people to understand their assets and problems and to create 
shared visions and hope for the future. Using data, people from different 
perspectives can often find common ground, agreeing on the results they want to 
achieve, their priorities for action and the indicators that will demonstrate joint 
progress toward improving the well-being of the children, youth and families in 
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their communities. Data-driven decision-making processes can also help decision- 
makers hold people accountable for the results that want to achieve. (McCroskey 
1999: 1) 

Tools to facilitate the effective use of data and help tell the story of Proposition 10 

The following are specific tools that can be employed to facilitate the effective use of data. 
Proposition 10 commissions may want to invest in these and other tools which would be helpful in 
planning for and understanding the impact of Proposition 10: 

> Data matching, or electronic matching of individual case files to determine the 
number of cases served by more than one system, can be a useful technique to 
better understand where the highest needs for services may be, where to locate new 
facilities and where to focus resources. 

> Resource mapping or asset mapping uses geographic information systems (GIS) 
to map locations of facilities and informal resources, identify administrative 
districts, and overlay population factors. 

> Children ’s budgets provide analysis of annual state, county, municipal and other 
expenditures targeted to children and families. 

> Projections, or trend analyses, assess change in key indicators over time, tracking 
past rates of change to make predictions about future conditions. 

> A common set of definitions, or data dictionary, can provide some uniformity 
across data systems. Although many systems will not be able to change definitions 
immediately (especially if definitions have been mandated by funders), a shared 
direction for the future can be extremely helpful. 

> Net-based information sharing provides immediate access to key data through the 
Internet rather than requiring exchange of disks or paper reports. 

> Reporting key indicator data both countywide and in small geographic area 
reports will help community partners find the data most relevant to their own 
planning and service delivery. For example, reports could include data by school 
attendance areas, zip codes and census tracts. 

Need for training 

Because many of these uses of data are relatively new, ongoing training will be required for planners, 
program providers, community members and decision-makers. To begin with, training is needed to 
help commissioners and their community partners develop a shared vocabulary and processes to 
inform results-based decision-making. Involvement of community members in all data-related 
training activities is essential to assure trust, facilitate communication and encourage collaborative 
decision-making. 
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V. Recommendations on Funding and Using Results-Based Decision Making 



Proposition 10 commissions across the state will need to develop results-based planning and 
accountability processes quickly in order to assure commissioners and the public that funds are being 
well used. The following are our recommendations on how to get where we need to be as quickly 
and efficiently as possible. 

1. Build on existing systems. Using available data whenever possible makes sense both for 
programmatic and for financial reasons. Collaborative approaches to sharing data and improving 
existing systems will benefit not only Proposition 10, but will enhance the entire child and family 
service system. 

2. Integrating existing data systems is a strategic investment. A strategic-investment approach 
would suggest that commissions invest in improving databases that are shared across systems, 
including development of common definitions. Internet-based information sharing, data matching, 
small-area data reporting and geographic information systems. For example, all county health 
departments must collect select maternal and child health indicators for state reporting requirements. 
These data can be better integrated - for example, birth certificates linked to prenatal risk 
assessment. Similarly, birthweight data could be linked to school reference data for years later. 

3. Invest in a countywide and statewide data infrastructure. Whether formal or informal, a 
collaborative group focused on data should include representation from the many groups that can 
help turn reams of data into useful information. Where such an infrastructure does not exist. 
Proposition 10 commissions should invest some portion of their funds to develop a data group to 
inform ongoing planning and monitoring of funded services. The State Commission can help by 
involving the state departments most concerned with young children and their families in a parallel 
statewide effort. It will be essential for the State Commission to take a leadership role in data 
generation and/or in helping counties develop the capacity to collect data for themselves. Because 
many smaller counties either do not have the capacity to do adequate data collection or are not aware 
of what is needed to build their capacity, the state should consider developing a clearinghouse for 
state and county data, “best/promising practices,” and technical assistance around data issues. 

4. Invite investment from local funders. Local foundations and businesses may be willing to 
supplement investment in these data efforts if they see the long-term advantages of having better data 
about what works for children and families. 

5. Develop a research agenda. Commissions should use their experiences to create research 
agendas that will help them develop better indicators, address unanswered questions and suggest 
directions for the future. Many local researchers would be glad to focus their inquiries on questions 
relevant to local decision-makers, if they knew the key questions that commissioners and other 
decision-makers were seeking to answer. Partnerships with higher education and community-based 
research groups could provide more information to help children and families sooner than could 
commissions acting on their own. 
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6. Share lessons learned with other commissions (the State Commission and county commissions) 
and with the public . The most difficult lessons to share are not about what worked, but about what 
didn’t work. No one likes to highlight one’s mistakes, but if we admit that we all make them, it will 
become easier to share experiences and to work together to build a better future for all the children 
of California. 



VI. Appendix A: Language Translation Key 



PROPOSITION 
10 GUIDELINES 
TERMS 


OTHER 

COMMONLY USED 
TERMS 


PROPOSITION 10 
GUIDELNES 
DEFINITION 


LEVEL OF 
MEASUREMENT 


EXAMPLE 


GOAL/ 

STRATEGIC 

RESULT 


DESIRED OUTCOME/ 
RESULT 


A long-term (5-10 
years) statement of 
desired change based 
upon the vision 
statement. 


Whole population 


Healthy children 


OBJECTIVE 


TARGET 


A precise description 
of desired change that 
is short-range and 
measurable and that 
supports the 
achievement of the 
goal. 


Whole population 


The number of 
children bom drug- 
exposed in X county 
will be reduced to Y by 
2001. 


OUTCOMES 


INDICATORS 


The actual measure of 
the extent to which 
results are being 
achieved. 


Whole population 


The number of 
reported substance- 
exposed births in X 
County in a given year. 


STRATEGIES 


STRATEGIES 


The course of action 
taken to achieve stated 
goals and objectives. 




Create or expand 
residential treatment 
services for substance- 
abusing pregnant 
women in X County. 


INDICATORS 


PERFORMANCE 

MEASURES 


Specific process or 
performance measures 
that are used to 
determine whether 
programs, services or 
projects are achieving 
goals and objectives. 


Program 

target population 


Quantity: Number of 
women served by a 
residential treatment 
program for substance 
abusing pregnant 
women. 

Quality: Percent of 
babies bom drug-free 
among the women 
participating in the 
treatment program. 



Source: Sutherland, C, Shulman, E, Halfon, N, Gardner, S, McCroskey, J, From Resources to Results for 
California ’s Children and Families: A Guide to Proposition 10 Strategic Planning and Implementation, UCLA 
Center for Healthier Children, Families and Communities, 2000. 




VII. Appendix B: Examples of Indicators of Short-Term Results from the State 
Commission “Results” Document 



I. Long-Range Strategic Result: Improved Family Functioning/Strong Families 

Short-term result: 

1. Increased capacity to provide parenting education 

Example indicators: 

- Number of parent education classes/opportunities available either in the classroom or 
delivered through other means (e.g., home visitation, etc.) 

- Number of parents/prospective parents participating in parent education classes and 
other educational opportunities 

2. Increased capacity to provide parent support services 

Example indicators: 

- Number of families served by home visitation programs 

- Number of families served in family resource centers 

II. Long-Range Strategic Result: Improved Child Development/Children Learning And Ready 
for School 



Short-term result: 

1. Increased capacity to serve infants and children in licensed and accredited child care facilities 

Example indicators: 

- Number of accredited family child care homes and child care centers 

- Availability of child care options for families with alternative work schedules 

2. Increased supports and educational opportunities for all child care providers 

Example indicators: 

- Access to and support for unit-bearing continued education courses and training for 
child care providers 

- Establishment of incentive/compensation programs linked to higher levels of training 
and longevity 



3. Increased access to licensed family child care and center-based child care 

Example indicator 

- Number of child care resource and referral services 

4 . Increased capacity to serve children with special needs 

Example indicator 

- Number of children with developmental delays and other special needs who have access 
to quality child care programs in typical environments 

5. Increased readiness for kindergarten 

Example indicators 

- Number of child care providers receiving training about school readiness 

- Number of children who receive mental health and developmental screenings and 
appropriate referral services 

- Availability and use of family literacy programs 

III. Long-Range Strategic Result: Improved Child Health/Healthv Children 

Short-term result: 

1. Increased health service capacity in Proposition 10 target areas (such as perinatal services, 
immunizations, mental health assessment, child health screenings) 

2. Increased maternal access to perinatal health care services 

Example indicators 

- Number of women enrolled in existing or new programs providing prenatal services to 
women 

- Percentage of infants bom with healthy birthweights 

3. Increased child access to health, mental health and dental services 

Example indicators 

- Percent of children with up-to-date immunizations at age 2 and at kindergarten entry 

- Number of children with dental caries, especially untreated dental caries 
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4. Increased child access to early screening and early intervention for developmental delays and 
other special needs 

Example indicator 

- Number of referrals of children to existing or new services for screening and early 
intervention for developmental delays and other special needs through expanded 
interagency relationships and training 

5. Increased family access to smoking, alcohol and substance abuse cessation/treatment 
supports 

Example indicator 

- Number of pregnant women accessing smoking, substance-abuse or alcohol treatment 
services 

6. Increased child access to good nutrition and exercise 

Example indicators 

- Number of infants who are breastfeeding 

- Community offerings of affordable and accessible activities promoting physical activity 
for families with young children 

7. Increased child access to healthy and safe environments 

Example indicators 

- Number of families who use tobacco in their homes 

- Number of accidents related to guns 



For a complete list of suggested indicators, see California Children and Families Commission. 
(1999). Results . 



VIII. Appendix C: Matrix of Existing Child Well-Being Indicators 



INDICATOR 


America *s 
Children 
(national,) 


KIDS 

COUNT 

(states) 


Children 

Now 

(Calif.) 


Foundation 

Consortium 


Los 

Angeles 

County 

Scorecard 


Santa 

Barbara 

County 

Scorecard 


Orange 

County 

Annual 

Report 


Economic Security 
















% children in poverty 


X 


X 


X 


X 


X 


X 




% children in extreme 
poverty (below 50% of 
poverty) 




X 












Median family income 




X 




X 








% children receiving free- 
reduced school lunch 
program 








X 


X 


X 


X 


Child support orders and 
collections per cases 




X 




X 






X 


% families receiving 
Temporary Assistance to 
Needy Children 
(TANFyCal WORKs 








X 


X 


X 


X 


% of births to unmarried 

mothers/single-parent 

families 




X 




X 




X 




% of children under age 1 8 
living with parents with at 
least one parent employed 
full-time all year 


X 














% of children living with 
parents who do not have 
full-time, year-round 
employment 




X 












% of households with 
children under age 1 8 that 
report any of three housing 
problems 


X 




X 










% of children under age 18 
in households experiencing 
food insecurity with 
moderate or severe hunger 


X 














% of births covered by 
Medicaid 




X 












% of children under age 18 
covered by health insurance 


X 


X 


X 




X 






% of children covered by 
Medicaid or other public- 
sector health insurance 




X 












% of children under age 18 
with no usual source of 
health care 


X 














Child care costs 






X 


X 




X 


X 


# of licensed child care 
spaces per 100 children 








X 


X 






# of request for child care 












X 


X 


Services to homeless 
children 












X 
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INDICATOR 


America *s 
Children 
(national,) 


KIDS 

COUNT 

(states) 


Children 

Now 

(Calif.) 


Foundation 

Consortium 


Los 

Angeles 

County 

Scorecard 


Santa 

Barbara 

County 

Scorecard 


Orange 

County 

Annual 

Report 


Health 
















% of children under age 1 8 
in very good or excellent 
health 


X 














% of children ages 5 to 1 7 
with any limitation in 
activity resulting from 
chronic conditions 


X 














Number of children who 
have been identified as 
having a developmental 
disability and utilized 
services 














X 


Live birth rate per 1,000 
population 








X 








% of low-birthweight 
infants (weighing less than 
5.5 pounds/2,500 grams at 
birth) 


X 


X 


X 


X 


X 


X 


X 


Infant mortality (deaths in 
the first year of life) per 
1,000 live births 


X 


X 


X 


X 


X 




X 


Maternal death rate per 
100,000 live births 








X 








# of substance-exposed 
infants bom 














X 


% of pregnant women 
receiving prenatal care in 
the 1“ trimester 






X 


X 








% of children ages 19 to 35 
months who received 
combined series 
immunization coverage 


X 


X 






X 


X 


X 


% of kindergarten students 
who have not been 
adequately immunized 








X 








Number of participants 
served by supplemental 
food/nutrition program 














X 


% of children age 2 to 5 
with a good diet 


X 














% of children with iron 
deficiency anemia 








X 








Deaths per 1 00,000 
children 1 to 4 (or 1 to 14 
or 0 to 1 7) 


X 


X 




X 


X 






Tuberculosis cases reported 










X 


X 




HIV/AIDS cases reported 










X 






Births per 1 ,000 females 
ages 15 to 17 (or 15-19) 


X 


X 




X 


X 


X 


X 


% of teen births to mothers 
who smoked 




X 












Children served by the 
Mental Health/Behavioral 
Health Department 










X 


X 


X 


Good air quality days - 
ozone standard met 










X 








A O 

l ± (— 



INDICATOR 


America *s 
Children 
(national,) 


KIDS 

COUNT 

(states) 


Children 

Now 

(Calif) 


Foundation 

Consortium 


Los 

Angeles 

County 

Scorecard 


Santa 

Barbara 

County 

Scorecard 


Orange 

County 

Annual 

Report 


Education 
















% of births to women who 
have not completed high 
school 








X 








% of children ages 3 to 5 
who are read to every day 
by a family member 


X 














% of children ages 3 to 4 
who are enrolled in 
preschool 


X 














% of eligible children 
enrolled in Head Start 






X 










% Limited-English- 
Proficient students 








X 






X 


Average mathematics scale 
score of 9-year-olds 


X 




X 


X 






X 


Average reading scale score 
of 9-year-olds 


X 


X 


X 


X 






X 


% of young adults 1 8 to 24 
who have completed high 
school/high school 
graduation rate 


X 








X 


X 




% of teens who are high 
school dropouts 




X 




X 




X 


X 


Safety 
















Confirmed child abuse 
cases/ cases opened 






X 


X 


X 


X 


X 


Deaths related to child 
abuse 






X 










Infants and children in 
foster care 






X 


X 


X 


X 


X 


Adoptions: children placed 










X 




X 


Children and women using 
emergency shelter services 












X 




Gun-related injuries and 
deaths/homicide rates 
among youths under 1 8 






X 




X 






Rate of serious violent 
crime victimizations per 
1 ,000 youths age 12-17 


X 














Serious violent crime 
offending rate/arrests per 
1 ,000 youths age 1 2 to 1 7 


X 


X 




X 


X 




X 


Juvenile in probation 
caseloads and out-of-home 
placements 












X 




Motor vehicle accident 
injuries and deaths among 
children under age 6 






X 




X 







Sources: 

America’s Children: Key National Indicators of Well-Being, 1999. 

KIDS COUNT, Annual. 

Children Now Report Card, Annual. 

Foundation Consortium. 

United Way of Greater Los Angeles and Los Angeles County Children’s Planning Council, 1999. 
Kids Network and the University of California, Santa Barbara, Graduate School of Education, 1998. 
Orange County Children’s Services Coordination Commission, 1999. 
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IX. Appendix D: Data Development Agenda 



Two organizations — Children Now and the California Report Card/County Databook Advisory 
Committee — believe that the following data are essential to future planning for young children and 
their families at the county level, yet are currently unavailable: 

> The number of children waiting for child care spaces 

> The availability of high-quality, affordable child care 

> Children’s preschool attendance 

> Number of children without health insurance 

> Number of toddlers up to date on their immunizations 

> Children’s dental health 

> Cases of asthma, developmental delays and other prevalent conditions for young children 

> Children’s housing status and conditions 

> Parent education levels and employment status 

> Rates of maltreatment of young children 

Other kinds of data that would be useful to further our understanding of how well children and 
families are doing include: 

> Rates of maternal depression 

> Number of babies bom substance-exposed 

> Number of parents reading to children weekly 

> School-readiness at age 5 

> Language abilities through age 3 

> Food security of families with young children 

> % of family income spent on housing/adequacy of housing 

> Homeless families with young children 

> Rates of preventable childhood diseases and disabilities 



X. Appendix E: Examples of Indicators Related to Service Integration, 
Accessibility and Cultural Appropriateness Presented in the State Commissions 
Results Document 

Increased Service Integration 

> Joint community planning efforts and decisions on revenue maximization and fund 
allocations. 

> Number of agencies and groups collaborating to provide integrated services who 
have signed Memorandums of Understanding with core players outlining their 
goals and roles. This includes addressing such issues as facilitation of entry into 
the service system, coordinated service delivery, shared information and non- 
duplication of effort. 

> Number of service providers with interdisciplinary training. 

Increased Accessibility of Services 

> Number of families who access services via conveniently located service sites, 
sites that co-locate multiple service providers, and/or multidisciplinary home- 
based services. 

> Availability of services to working families through flexible scheduling. 

> Availability of quality services to families of children with special needs through 
training of service providers (such as child care) to serve children with special 
needs or expanding the capacity of providers with expertise in special needs to 
meet a broader range of needs. 

Increased Cultural Competence in Service Provision 

> Numbers and types of child and family services available in locally appropriate 
languages other than English. 

> Numbers of service providers who are ethnically and culturally reflective of their 
communities. 

> Numbers and types of cultural diversity training (such as ESL, culturally specific 
food ways and healthcare practices) available and delivered to providers of 
services to children and families. 

Increased Public Engagement in Policy 

> Number of service recipients participating on policy boards and in program 
implementation. 

> Number and frequency of public input opportunities (such as hearings, focus 
groups and surveys). 

> Availability of and parent participation in public policy and advocacy training. 



XI. Appendix F: Principles for Using Outcomes and Indicators 



(Adapted from Outcome Measurement for Family and Children’s Services in Los Angeles County, 

drafted for the Los Angeles County Children’s Planning Council by J. McCroskey, August 1992.) 

1. Outcomes should be expressed as positive expressions of child and family well-being rather than 
absence of negative conditions (i.e., good health rather than decreased illness). Many of the 
indicators which measure those outcomes based on available data will, however, be phrased in 
the negative because that is how data are currently collected. 

2. Since no one indicator captures the full dimensions of outcomes sought, outcomes should be 
measured by a small set of indicators based on valid and reliable data from multiple sources. 
(Commissions should also recognize that trying to change too many indicators at once will dilute 
the possible impact of Proposition 10 funds.) 

3. Initial efforts should focus on a strategic set of outcomes and indicators that reflect concerns 
shared by the entire community. 

4. The process of developing practical and appropriate outcomes and indicators that accurately 
reflect the conditions of children and families is an evolutionary one, from which there is much 
to learn. Clarification of the cultural and community values which underlie the process is 
essential. 

5. The process should include experimentation with technical matters, including assessment of the 
accuracy of available data, sensitivity of indicators to change over time, identification of proxy 
indicators that reflect desired outcomes, and improvements in existing information systems. The 
inclusion of multiple disciplines and perspectives is crucial since no one group is expert in all 
of the substantive areas which must be combined to reflect overall child and family well-being. 
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XII. Appendix G: Resources 



Websites and Online Resources 

America *s Children: Key National Indicators of Well-Being, 1999. 

HTML browsable and PDF copies: http://childstats.gov 

Printed copies: National Maternal and Child Health Clearinghouse at (703) 356-1964 or nmchc@circsol.com for a 
free copy; or, if free copies run out, the Government Printing Office (hhtp://www.gpo.gov or (202) 512-1800, 
publication # 065-000-01 162-0) 

California Health Interview Survey (CHIS) 

UCLA Center for Health Policy Research 
10911 Weybum Avenue, Suite 300 
Los Angeles, CA 90024 
Email: chis@ucla.edu 
Website: www.healthpolicy.ucla.edu 

The Child Indicator: The Child, Youth and Family Indicators Newsletter, 
do Child Trends 

4301 Connecticut Ave., NW, Suite 100 
Washington, DC 20008 
childindicator@childtrends.org 

Children Now 

1212 Broadway, 5th Floor 

Oakland, CA 94612 

Phone: (5 10) 763-2444 

Fax: (510) 763-1974 

Email: children@childrennow.org 

Website: http://www.childrennow.org 

Connect for Kids 

Website: http://www.connectforkids.org/listservl 579/listserv.htm 

Federal Interagency Forum on Child and Family Statistics 
Website: http://childstats.gov 

KIDS COUNT 

do Annie E. Casey Foundation 
701 St. Paul St. 

Baltimore, MD 21202 

Phone: (410) 547-6600 

Fax: (410) 547-6624 

Website: http://www.aecf.org/kidscount 

National Neighborhood Indicators Project (NNIP) 

The Urban Institute 
2100 M Street, NW 
Washington, DC 20037 
Website: http://www.urban.org/nnip 

The Results and Performance Accountability Implementation Guide 
Website: http://www.RAGuide.org 
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Trends in the Well-Being of America's Children and Youth 

Website: http://aspe.hhs.gov/hsp/98trends/trends98.htm or call (202) 619-0257. 

The Work Sampling System (a kindergarten school -readiness assessment) 

(Developed by S. Meisels, J. Jablon, D. Marsden, M. Dichtelmiller, and A. Dorfman) 
Website: http://www.rebusinc.com/index2.html 



Publications 

County of San Diego Health and Human Services Agency. San Diego County Child and Family Health and Well- 
Being Report Card. San Diego: Author, 2000. 

Fielding JE and Sutherland CE. National Directory of Community Health Report Cards. Chicago: Health Research 
and Education Trust, 1998. 

Kids Network and the University of California, Santa Barbara, Graduate School of Education. Santa Barbara County 
Children s Scorecard. Santa Barbara: Author, 1998. 

Smith M. Program Management/Evaluation Support Systems for County Children and Families Commissions: An 
Assessment of Technology and Service Options. The Proposition 10 Technical Assistance Service Center, Center for 
Health Improvement, June 2001. 

Sutherland C, Shulman E, Halfon N, Gardner S, McCroskey J. From Resources to Results for California ’s Children 
and Families: A Guide to Proposition 10 Strategic Planning and Implementation, UCLA Center for Healthier 
Children, Families and Communities, 2000 

Sutherland C and Fielding JE. Creating Effective Community Health Report Cards. UCLA Center for Healthier 
Children, Families and Communities, 2001. 



Relevant Experts/Organizations 

The Alameda County Children and Families Commission 

Mark Friedman, Executive Director 

1850 Fairway Drive 

San Leandro, CA 94577 

Phone: (510) 667-7575 

Fax: (510)614-8850 

Email: mfnedma@co.alameda.ca.us 

Website: http://www.ackids.org 

Center for Collaboration for Children 
California State University Fullerton 
Sharon Milbum, Ph.D. 

Department of Child and Adolescent Studies 

College of Human Development and Community Service 

Phone:(714)278-2930 

Greg Robinson, PhD, Director 
Social Science Research Center 
Phone: (714) 278-2600 
Website: http://www.ecena.org 



The California Children and Families Association 
Contact: Steve Ladd 

Kern County Children and Families Commission 

2724 L Street 

Bakersfield, CA 93302 

Phone: (866) To-CCAFA 

Email: info@ccafa.org 

Website: http://www.ccafa.org 

The California Children and Families Commission 

501 J Street, Suite 530 

Sacramento, CA 95814 

Phone: (916)323-0056 

Fax: (916)323-0069 

Website: http://www.ccfc.ca.gov 

Corporation for Standards and Outcomes (CS & O) 

65 Enterprise 

Aliso Viejo, CA 92656 

Phone: (800) 587-7861 

Fax: (949)330-8081 

Website: http://www.csando.com 

Family Health Outcomes Project 
University of California, San Francisco 
Gerry Oliva 

3333 California Street, Suite 365 
San Francisco, CA 94118 
Phone: (415) 476-5283 
Fax: (415) 502-0848 
Email: fhop@itsa.ucsf.edu 
Website: http://www.ucsf.edu/fhop 

The Finance Project 

1000 Vermont Avenue NW 

Washington, DC 20005 

Phone: (202) 628-4200 

Website: http://www.financeproject.org 

The Fiscal Policy Studies Institute 
8 Charles Plaza, Suite 1407 
Baltimore, MD 21201 
Phone: (410) 659-9745 
Fax: (410) 659-9753 
Email: xfpsi@aol.com 

Website: http://www.resultsaccountability.com 

Foundation Consortium 
2295 Gateway Oaks Drive, Suite 100 
Sacramento, CA 95833 
Phone: (916) 646-3646 

Website: http://www.foundationconsortium.org 
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Sid Gardner, M.P.A. 

President 

Children and Family Futures 

4940 Irvine Blvd., Suite 202 

Irvine, CA 92620 

Phone: (714) 505-3525 

Fax: (714) 505-3526 

Email: sidgardner@home.com 

Website: http://www.cffutures.com/index.html 

Healthy Start Field Office and the California Center for Community-School 

Partnerships 

CRESS Center 

University of California, Davis 

1 Shields Avenue 

Davis, CA 95616-8729 

Phone: (530) 752-1277 or (530) 754-6343 

Website: http://hsfo.ucdavis.edu/ccc-sp.html 

Jacquelyn McCroskey 
Associate Professor 
USC School of Social Work 
Montgomery Ross Fisher Building 
Los Angeles, CA 90089-041 1 
Phone: (213) 740- 2004 
Fax:(213)740-0789 
Email: mccroske@rcf-fs.usc.edu 

Sam Meisels 

Professor, School of Education 
University of Michigan 
School of Education 
3210 SEB 

Ann Arbor, MI 48109 
Phone: (734) 647-0621 
Email: smeisels@umich.edu 

Proposition 10 Technical Assistance Center 
California Center for Health Improvement 
1330 21st Street, Suite 100 
Sacramento, CA 95814 
Phone: (916) 930-9200 
Fax: (9 16) 930-9010 

E-mail: Prop 10TA@centerforhealthimprovement.org 
Website: http://www.cchi.org/cgi-bin/cchi/default.asp 

The Search Institute 
700 South 3 rd Street, Suite 210 
Minneapolis, MN 55415 
Phone: (612)371-8955 

Website: http://www.search-institute.org/assets 
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UCLA Center for Healthier Children, Families and Communities 
10945 Le Conte Avenue, Suite 1401 
Box 956939 

Los Angeles, CA 90095-6939 
Phone: (310) 825-8042 
Fax: (310) 206-3180 
Website: http://healthychild.ucla.edu 

UCSF Child Services Research Group 
44 Montgomery Street, Suite 1450 
San Francisco, CA 94104 
Phone: (415) 502-6174 
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XTTT. Appendix H: The Multiple Determinants of Health Model 



CD 
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cv 
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FIGURE 1 : THE MULTIPLE DETERMINANTS OF HEALTH MODEL. “Multiple determinants” models of health production illustrate the different 
kinds of factors (eg. social, physical, genetic) that determine health, and the pathways through which these factors create health. Multiple determinants 
models have played an important public policy role, broadening the range of issues that are understood to be health issues, and illustrating the 
relationships between environmental, social and economic factors. Adapted from: Evans, Robert G., et al. [eds.J Why Are Some People Healthy and 
Others Not? The Determinants of Health Populations . New York: Aldine de Gruyter. Copyright 1994, Walter de Gruyter Inc., New York. 



XIV. Appendix I: Influence of Risk-Reduction and Health Promotion Strategies on Health Development 
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